Data processing systems with distributed architectures have become increasingly popular, particularly following the widespread diffusion of the Internet. In a distributed architecture, client computers consume services offered by server computers across a network. Two or more servers can be grouped into a cluster, so as to appear as a single computer to the clients. The cluster provides a single point of management and facilitates the scaling of the system to meet increasing demand. There are various clustering techniques that facilitate clustering of servers. One such clustering technique is the load-balancing model.
The load-balancing attempts to optimize the distribution of workload across the servers. Particularly, in a cluster of the network load balancing type incoming requests from clients can be distributed across servers that share a single (virtual) network address.
In some other clustering techniques, different clusters serve different types of requests, e.g., a first set of servers serves users who access a first version of the application and a second set of servers serves users who access a second version of the application. Load balancing between such clusters is different from the above load balancing solution and is typically more complex, because access requests from clients cannot be distributed between clusters as the clusters serve different types of requests. Current load balancing solutions balance the load between such clusters by moving resources, e.g., servers, between the clusters.
The current load balancing solutions typically use a metric, e.g., central processing unit (CPU) idle time, to move servers from a cluster that has high CPU idle time to a cluster that has low CPU idle time. CPU idle time is the amount of time, e.g., during a measurement time period, during which the CPU is not utilized or has low utilization. However, such load balancing solutions are typically inefficient. One reason why such load balancing solutions are typically inefficient is because in some situations, e.g., when there are memory issues on web servers, the CPU idle may not actually affect the performance of the cluster. While there are memory issues in the server, the CPU idle may look acceptable and therefore, the load balancing solution may let the server be moved to another cluster, thereby causing the performance of the cluster to deteriorate from bad to worse. Such load balancing solutions can create an imbalance rather than balance in the resource utilization in the distributed system.